Leaving Local Optima in Unsupervised Kernel Regression
نویسندگان
چکیده
Abstract. Embedding high-dimensional patterns in low-dimensional latent spaces is a challenging task. In this paper, we introduce re-sampling strategies to leave local optima in the data space reconstruction error (DSRE) minimization process of unsupervised kernel regression (UKR). For this sake, we concentrate on a hybrid UKR variant that combines iterative solution construction with gradient descent based optimization. Patterns with high reconstruction errors are removed from the manifold and re-sampled based on Gaussian sampling. Re-sampling variants consider different pattern reconstruction errors, varying numbers of resampled patterns, and termination conditions. The re-sampling process with UKR can also improve ISOMAP embeddings. Experiments on typical benchmark data sets illustrate the capabilities of strategies for leaving optima.
منابع مشابه
Covariance Matrix Self-Adaptation and Kernel Regression - Perspectives of Evolutionary Optimization in Kernel Machines
Kernel based techniques have shown outstanding success in data mining and machine learning in the recent past. Many optimization problems of kernel based methods suffer from multiple local optima. Evolution strategies have grown to successful methods in non-convex optimization. This work shows how both areas can profit from each other. We investigate the application of evolution strategies to N...
متن کاملEvolutionary kernel density regression
The Nadaraya–Watson estimator, also known as kernel regression, is a density-based regression technique. It weights output values with the relative densities in input space. The density is measured with kernel functions that depend on bandwidth parameters. In this work we present an evolutionary bandwidth optimizer for kernel regression. The approach is based on a robust loss function, leave-on...
متن کاملEvolutionary Unsupervised Kernel Regression
Dimension reduction and manifold learning play an important role in robotics, multimedia processing and data mining. For these tasks strong methods like Unsupervised Kernel Regression [4, 7] or Gaussian Process Latent Variable Models [5, 6] have been proposed in the last years. But many methods suffer from numerous local optima and crucial parameter dependencies. We use advanced methods from st...
متن کاملUsing Particle Swarm Optimization and Locally-Tuned General Regression Neural Networks with Optimal Completion for Clustering Incomplete Data Using Finite Mixture Models
In this paper, a new algorithm is presented for unsupervised learning of Finite Mixture Models using incomplete data set. This algorithm applies Particle Swarm Optimization to solve the local optima problem of the Expectation-Maximization algorithm. In addition, the proposed algorithm uses Locally-tuned General Regression neural networks with Optimal Completion Strategy to estimate missing valu...
متن کاملTHE COMPARISON OF TWO METHOD NONPARAMETRIC APPROACH ON SMALL AREA ESTIMATION (CASE: APPROACH WITH KERNEL METHODS AND LOCAL POLYNOMIAL REGRESSION)
Small Area estimation is a technique used to estimate parameters of subpopulations with small sample sizes. Small area estimation is needed in obtaining information on a small area, such as sub-district or village. Generally, in some cases, small area estimation uses parametric modeling. But in fact, a lot of models have no linear relationship between the small area average and the covariat...
متن کامل